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1 1  abstract  (Maximum  200  words) 

The  major  problem  for  the  present  contract  period  was  to  extend  fast  algorithms 
based  on  displacement  structure  to  matrices  with  zero  minors  -  the  socalled 
singular  case.  Almost  all  the  literature,  of  about  a  hundred  years,  deals  with 
the  socalled  regular  or  nonsingular  cases,  with  particular  success  in  the  case 
of  Hankel  and  Hankel-related  matrices.  These  results  are  related  to  the  now 
we 11 known  Berlekamp-Massey  algorithm  (for  solving  Hankel  linear  equations) . 

For  Toeplitz  and  Toeplitz-related  matrices,  there  were  only  some  partial  and  rather 
complicated  solutions.  In  the  Ph.D.  research  of  D.  Pal  a  complete  and  elegant 
solution  is  given  to  this  problem  for  the  case  of  Toeplitz  and  quasi-Toeplitz 
matrices.  While  not  as  general  as  one  would  have  liked,  the  latter  class  of 
matrices  allowed  one  to  get  the  first  general  solution  to  the  much-studied 
stability  and  root-distribution  problems  for  discrete-time  systems.  Additional 
results  appear  in  the  list  of  publications  in  the  appendix. 
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This  is  the  final  report  on  ARO  Contract  DAAL-89-K-0109,  May  16,  1989  -  May 
15,  1992. 

A  substantial  amount  of  work  was  accomplished  during  the  contract  period,  as  may 
be  seen  from  the  list  of  publications  in  the  Appendix.  Of  course,  because  of  the  various 
delays  and  lags  in  the  publications  process,  several  of  the  papers  relate  to  work  initiated 
and  or  completed  in  earlier  contract  periods. 

The  major  problem  posed  in  our  research  proposal  for  the  present  contract  period 
was  to  extend  fast  algorithms  based  on  displacement  structure  to  matrices  with  zero 
minors  -  the  socalled  singular  case.  Almost  all  the  literature,  of  about  a  hundred  years, 
deals  with  the  socalled  regular  or  nonsingular  cases,  with  particular  success  in  the  case 
of  Hankel  and  Hankel- related  matrices.  These  results  are  related  to  the  now  wellknown 
Berlekamp-Massey  algorithm  (for  solving  Hankel  linear  equations). 

For  Toeplitz  and  Toeplitz-related  matrices,  there  were  only  some  partial  and  rather 
complicated  solutions.  In  the  Ph.D.  research  of  D.  Pal  we  have  given  a  complete  and 
elegant  solution  to  this  problem  for  the  case  of  Toeplitz  and  quasi-Toeplitz  matrices. 
While  not  as  general  as  we  would  have  liked,  the  latter  class  of  matrices  allowed  us  to 
get  the  first  general  solution  to  the  much-studied  stability  and  root-distribution  problems 
for  discrete-time  systems.  A  paper  on  this  last  result  has  been  accepted  by  the  IEEE 
Transactions  on  Automatic  Control ,  while  a  paper  on  the  underlying  mathematics  has 
been  accepted  by  the  SIAM  Journal  on  Matrix  Analysis.  These  researches  spurred 
related  studies  on  stability  and  root  distribution  problems,  with  a  long  paper  giving  a 
unified  approach  to  half-plane  (or  continuous-time  system)  and  unit  disc  (discrete-time 
system)  problems  appearing  in  the  February  1991  issue  of  the  IEEE  Transactions  on 
Circuits  and  Systems.  In  that  paper,  we  had  studied  only  the  regular  (or  nonsingular) 
case,  but  were  able  to  show  that  specializing  our  earlier  general  fast  algorithms  for 
matrices  with  displacement  structure  allowed  us  to  get  a  natural  approach  unifying  the 
Classical  Routh-Hurwitz  half-plane  tests  and  the  classical  Schur-Cohn  unit  disc  tests. 
Furthermore  it  showed  that  these  were  only  two  of  a  whole  family  of  different-looking, 
but  computationally  equivalent  tests.  We  also  obtained  a  framework  for  classifying  all 
possible  tests,  thus  for  example  including  some  recent  tests  of  Lepschy  et  al.  (1988).  In 
Pal’s  thesis  we  used  our  extension  of  the  displacement  structure  theory  to  handle  the 
singular  cases  of  the  stability/root  distribution  problem. 

Apart  from  the  above  major  effort  on  the  main  problem  put  forward  in  our  proposal, 
we  continued  to  build  upon  work  in  earlier  periods,  especially  extending  the  techniques 


of  Chun  (Ph.D.  thesis,  June  1989)  to  apply  to  other  classes  of  problems.  In  particular, 
Pal  used  these  ideas  to  find  new  ways  of  solving  linear  equations  with  singular  Toeplitz 
and  quasi-Toeplitz  matrices.  This  work  has  been  submitted  for  publication.  Recently 
with  another  student,  A.  Sayed,  the  ideas  have  been  applied  to  the  currently  much 
studied  QR  adaptive  filtering  algorithm.  We  have  thus  obtained  a  new  computational 
array  for  updating  the  weight  vector  in  the  adaptive  algorithm,  impoving  on  the  much- 
cited  McWhirter  array  (see,  e.g.  the  textbook  of  S.  Haykin,  Adaptive  Filter  Theory,  2nd 
edition,  Prentice-Hall  1991). 

In  a  related  effort,  we  made  some  connections  with  the  Ph.D.  studies  of  M.  Genos- 
sar  on  the  time-frequency  analysis  of  nonstationary  processes.  A  particularly  important 
subclass  of  such  processes  are  the  cyclostationary  processes,  whose  covariance  matri¬ 
ces  have  a  particular  kind  of  block-Toeplitz  structure.  In  the  course  of  studying  such 
processes,  we  made  a  complete  analysis  of  the  statistical  estimation  procedure  for  the 
socalled  cyclic  autocorrelation  (or  autocovariance)  matrix  of  such  processes.  We  have 
extended  the  earlier  work  of  Hurd  and  Gardner  on  this  problem;  a  paper  on  our  results 
is  to  appear  in  the  IEEE  Transactions  on  Information  Theory. 

Our  other  major  effort  has  been  on  fast  algorithms  for  subspace  computation  in  the 
new  high  resolution  sensor-array  processing  algorithms  such  as  MUSIC,  ESPRIT  and 
WSF.  In  the  thesis  work  of  G.  Xu  a  new  socalled  FSD  (Fast  subspace  decomposition) 
algorithm  was  developed  for  this  purpose.  This  led  us  to  see  how  the  ideas  of  displace¬ 
ment  structure  could  be  used  to  further  speed  up  the  FSD  computations.  Some  papers 
on  these  results  -  and  their  connections  to  certain  classes  of  eigenvalue  and  singular 
value  problems  -  are  under  review. 

Finally  we  may  mention  that  during  the  present  contract  period,  we  worked  on 
revising  and  resubmitting  papers  submitted  in  earlier  periods.  The  publications  list 
shows  that  this  has  been  a  major  activity.  Among  several  interesting  papers,  we  may 
make  mention  of  some  papers  coauthored  by  a  previous  postdoctoral  scholar,  A.  Dembo 
(see  [5], [6], [13], [17]  in  the  list  of  publications)  and  work  on  regular  iterative  arrays  for 
VLSI  design  (see  [4], [9], [12], [21]). 

The  Ph.D.  dissertations  completed  in  this  period  were  those  of  D.  Pal  (May  1990). 
G.  Xu  (Sept.  1991),  and  M.  Genossar  (April  1992);  abstracts  are  appended  to  this  report. 
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[2]  J.  Chun  and  T.  Kailath,  A  Constructive  Proof  of  the  Gohberg-Semencul  Formula,  Linear 
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ber  1989. 
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1990. 
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[10]  H.  Lev-Ari,  volume  22  of  Institute  for  Math,  and  Its  Appls.  chapter  Extension  Problems  Un¬ 
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Processing,  Part  I:  Signal  Processing  Theory  Edition,  1990. 
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Verlag,  New  York,  Signal  Processing,  Part  I:  Signal  Processing  Theory  Edition,  1990. 

[12]  J.  Chun  T.  Kailath  and  V.  Roychowdhury,  Systolic  Array  for  Solving  Toeplitz  Systems  of 
Equations,  pages  547-560,  Oxford  IBH  Publishing  Co.,  India,  Spectral  Analysis  in  One  or 
Two  Dimensions,  eds.  S.  Prasad  and  R.  Kashyap,  1990. 

[13]  A.  Dembo  and  0.  Zeitouni,  Maximum  A  Posteriori  Estimation  of  Elliptic  Gaussian  Fields 
Observed  Via  a  Noisy  Nonlinear  Channel,  J.  Multivariate  Anal.,  35(2):  151- 167,  Nov  1990. 


1 


[14]  J.  Chun  and  T.  Kailath,  Vol.  F70  of  NATO  ASI  Series,  Generalized  Displacement  Structure  for 
Block- Toeplitz,  Toeplitz-block,  and  Toeplitz-derived  Matrices,  pages  215-236,  Springer- Verlag, 
Berlin,  Numerical  Linear  Algebra,  Digital  Signal  Processing  and  Parallel  Algorithms,  1991. 

[15]  J.  Chun  and  T.  Kailath,  Divide-and-Conquer  Solutions  of  Least-Squares  Problems  for  Matrices 
with  Displacement  Structure,  SIAM  J.  Matrix  Anal,  and  Appl.,  12(1):128— 145,  January  1991. 

[16]  H.  Lev-Ari,  Y.  Bistritz,  and  T.  Kailath,  Generalized  (Bezoutians)  and  Families  of  Efficient 
Root-Location  Procedures,  IEEE  Trans.  Circuits  and  Systems,  38(2):170— 186,  February  1991. 

[17]  A.  Demboand  0.  Zeitouni,  Onsager-Machlup  Functional  and  Maximum  A  Posteriori  Estimation 
for  A  Class  of  Non-Gaussian  Random  Fields,  J.  Multivariate  Anal.,  36(2):243-262,  February 
1991. 

[18]  J.  Chun  and  T.  Kailath,  Displacement  Structure  for  Hankel,  Vandermonde  and  Related  (De¬ 
rived)  Matrices,  Linear  Algebra  Appl.,  151:199-227,  1991. 

[19]  J.  Chun  and  T.  Kailath,  Systolic  Array  Implementation  of  the  Square  Root  Chandrasekhar 
Filter,  Intemat.  J.  Control,  54(6):1385-1398,  December  1991. 

[20]  Y.  Bistritz,  H.  Lev-Ari,  and  T.  Kailath,  Immittance-Domain  Three-Term  Schur  and  Levinson 
Recursions  for  Quasi-Toeplitz  Complex  Hermitian  Matrices,  SIAM  J.  Matrix  Anal,  and  Appl., 
12(3):497~520,  July  1991. 

[21]  H.  V.  Jagadish  and  T.  Kailath,  Obtaining  Schedules  for  Digital  Systems,  IEEE 
Trans.  Acoust.  Speech  Signal  Process.,  39(10):2296-2316,  October  1991. 

[22]  T.  Kailath,  From  Kalman  Filtering  to  Innovations,  Martingales,  Scattering  and  Other  Nice 
Things,  pages  55-88,  Mathematical  System  Theory:  The  Influence  of  R  E.  Kalman,  ed.  A  C.  An- 
toulas.  Springer- Verlag,  Communications  and  Control  Engineering  Edition,  1991. 

[23]  T.  Kailath  Remarks  on  the  Origin  of  the  Displacement  Rank  Concept,  J.  Appl.  Math.  Compu¬ 
tation,  45(2):  193-206,  September  1991. 

[24]  H.  Lev-Ari  and  T.  Kailath,  State-Space  Approach  to  Factorization  of  Lossless  Transfer  Func¬ 
tions  and  Structured  Matrices,  Linear  Algebra  Appl.,  162-164:273-295,  February  1992. 

[25]  R.  Ackner,  H.  Lev-Ari  and  T.  Kailath,  Transmission-Line  Models  for  the  Modified  Schur 
Algorithm,  IEEE  Trans,  on  Circuits  and  Systems,  39(4):290-296,  April  1992. 

Papers  Accepted  for  Publication 

[26]  A.  Bruckstein,  T.  Kailath,  I.  Koltracht,  and  P.  Lancaster,  On  The  Reconstruction  of  Layered 
Media  From  Reflection  Data,  SIAM  J.  Matrix  Anal,  and  Appl. 

[27]  R.  Ackner  and  T.  Kailath,  On  The  Ptak- Young  Generalization  of  the  Schur-Cohn  Theorem. 
IEEE  Trans.  Automat.  Control. 

[28]  D.  Pal  and  T.  Kailath,  A  Fast-Matrix-Factorization  Approach  Towards  the  Singular  Cases 
of  Tabular  Form  Root  Distribution  Procedures:  The  Unit  Circle  Case,  IEEE  Trans.  Au¬ 
tomat.  Control,  1992. 

[29]  T.  Kailath  and  J.  Chun,  Generalized  Displacement  Structure  for  Block-Toeplitz,  Toeplitz-Block. 
and  Toeplitz-Derived  Matrices,  SIAM  J.  Matrix  Anal,  and  Appl. 


2 


[30]  R.  Ackner,  H.  Lev-Ari,  and  T.  Kailath,  The  Schur  Algorithm  for  Matrix- Valued  Meromorphic 
Functions,  SIAM  J.  Matrix  Anal,  and  Appl. 

[31]  D.  Pal,  Gohberg-semencul  type  formulas  via  embedding  of  lyapunov  equations,  IEEE 
Trans.  Acoust.  Speech  Signal  Process. 

[32]  M.  Genossar,  H.  Lev-Ari,  and  T.  Kailath,  Consistent  Estimation  of  Cyclic  Autocorrelation, 
IEEE  Trans.  Acoust.  Speech  Signal  Process. 

[33]  V.  P.  Roychowdhury,  L.  Thiele,  S.  K.Rao,  and  T.  Kailath,  On  the  Localization  of  Algorithms 
for  VLSI  Processor  Arrays,  IEEE  Trans.  Comput. 

Papers  Under  Review 

[34]  G.  Xu  and  T.  Kailath,  Fast  Algorithms  for  Computing  the  Principal  Singular  Values  and 
Vectors,  SIAM  J.  Matrix  Anal,  and  Appl. 

[35]  G.  Xu  and  T.  Kailath,  Fast  Decomposition  of  the  Principal  Eigenspace  Via  Exploitation  of 
Eigenvalue  Multiplicity,  SIAM  J.  Matrix  Anal,  and  Appl. 

[36]  D.  Pal  and  T.  Kailath,  Fast  Triangular  Factorization  and  Inversion  of  Hankel  and  Related 
Matrices  With  Arbitrary  Rank  Profile,  SIAM  J.  Matrix  Anal,  and  Appl. 

[37]  V.  P.  Roychowdhury  and  T.  Kailath,  Regular  Processor  Arrays  for  Matrix  Algorithms  with 
Pivoting,  ACM  Communications. 

[38]  Guanghan  Xu  and  Thomas  Kailath,  Fast  Signal-Subspace  Decomposition,  Pt.  I:  Ideal  Covari¬ 
ance  Matrices,  IEEE  Trans.  Acoust  Speech  Signal  Process. 

[39]  G.  Xu  and  T.  Kailath,  Fast  Signal-Subspace  Decomposition,  Pt.  II:  Sample  Covariance  Matrices, 
IEEE  Trans.  Acoust.  Speech  Signal  Process. 

[40]  D.  Pal  and  T.  Kailath,  A  Fast  Matrix  Factorization  Approach  Towards  the  Singular  Cases  of 
Tabular  Form  Root  Distribution  Procedures:  The  Imaginary  Axis  Case,  IEEE  Trans.  Circuits 
and  Systems ,  1991. 

[41]  G.  Xu,  S.  D.  Silverstein,  R.  H.  Roy,  and  T.  Kailath,  Beamspace  ESPRIT,  IEEE 
Trans.  Acoust.  Speech  Signal  Process. 

[42]  A.  Dembo,  A  New  Proof  of  the  Entropy  Power  Inequality,  IEEE  Trans,  on  Information  Theory 

[43]  M.  Goldburg,  M.  Genossar,  and  T.  Kailath,  Optimal  Coefficient  Selection  for  Discrete  Time 
Wavelet  Filters,  IEEE  Trans,  on  Signal  Processing 

[44]  G.  Xu,  R.H.  Roy,  and  T.  Kailath,  Detection  of  Number  of  Sources  Via  Exploitation  of  Centro- 
Symmetry  Property,  IEEE  Trans,  on  Signal  Processing 

Conference  Papers 

[45]  T.  Kailath  and  V.P.  Roychowdhury,  Scheduling  linearly  indexed  assignment  code,,  pages  118- 
129,  Los  Angeles,  CA,  January  1989.  Proc.  SPIE. 

[46]  D.  Pal  and  T.  Kailath,  Fast  modified  triangular  factorization  of  Hankel,  Quasi-Hankel  and 
Sign-Modified  Quasi-Hankel  matrices  with  arbitrary  rank  profile,  pages  614-618,  Asilomar, 
CA,  October  1989.  23rd  Asilomar  Conference  on  Circuits,  Signals  and  Computers. 


3 


[47]  T.  Varvarigou,  V.  P.  Roychowdhury,  and  T.  Kailath,  Some  new  algorithms  for  reconfiguring 
VLSI/WSI  arrays,  pages  229-235,  San  Francisco,  CA,  Jan  1990.  Intern’l.  Conference  on  Wafer 
Scale  Integration. 

[48]  D.  Pal  and  T.  Kailath,  A  generalization  of  the  Schur-Cohn  test:  The  singular  case,  Canada, 
August  1990.  33rd  Midwest  Symposium  on  Circuits  and  Systems. 

[49]  D.  Pal  and  T.  Kailath,  Approximate  AR  modeling:  An  alternative  to  regularization  via  Schur 
algorithm  for  matrices  with  arbitrary  rank  profile,  pages  275-279,  New  York,  October  1990. 
5th  Acoustics,  Speech  and  Signal  Processing  Workshop  on  Spectrum  Estimation  and  Modeling. 

[50]  G.  Xu  and  T.  Kailath,  Fast  signal  subspace  decomposition  without  eigendecomposition,  Asilo- 
mar,  CA,  November  1990.  24th  Asilomar  Conference  on  Circuits,  Signals  and  Computers,  966- 
969. 

[51]  G.  Xu  and  T.  Kailath,  A  simple  and  effective  algorithm  for  estimating  time  delay  of  communi¬ 
cations  signals,  Hawaii,  November  1990.  Int’l.  Symp.  on  Info.  Thy.  and  Its  Appls.,  267-270. 

[52]  D.  Pal  and  T.  Kailath,  Approximate  ar  modeling:  A  schur  approach,  Rochester,  NY,  October 
1990.  5th  ASSP  Workshop  on  Spectrum  Estimation  and  Modeling. 

[53]  G.  Xu  and  T.  Kailath,  A  Fast  Algorithm  for  Signal  Subspace  Decomposition  and  Its  Perfor¬ 
mance  Analysis,  pages  3069-3072,  Toronto,  Canada,  May  1991.  1991  Intern’l.  Conf.  Acoustics, 
Speech  and  Signal  Processing. 

[54]  G.  Xu,  S.  Silverstein,  R.  Roy,  and  T.  Kailath,  Parallel  implementation  and  performance  analysis 
of  beamspace  esprit,  Toronto,  Canada,  May,  1991  1991.  ICASSP-91. 

[55]  T.  Kailath,  Root  distribution  and  displacement  structure,  Chicago,  IL,  June  1992.  Symposium 
on  Fundamentals  of  Discrete-Time  Systems. 

[56]  Debajyoti  Pal  and  Thomas  Kailath,  A  systematic  generalization  of  the  schur-cohn  procedure 
for  root  localization  with  respect  to  the  unit  circle:  The  singular  cases,  Chicago,  IL,  June  1992. 
Symposium  on  Fundamentals  of  discrete-Time  Systems. 

[57]  G.  Xu,  R.  H.  Roy,  and  T.  Kailath,  Detection  of  source  number  with  centro-symmetric  arrays, 
1992  Intern’l.  Conf.  Acoustics,  Speech  and  Signal  Processing,  1992. 

Theses 

[58]  D.  Pal,  Fast  Algorithms  for  Structured  Matrices  With  Arbitrary  Rank  Profile,  Ph.D.  Disserta¬ 
tion,  Stanford  University,  May  1990. 

[59]  G.  Xu,  Fast  Subspace  Decomposition  and  Its  Applications,  Ph.D.  Dissertation,  Stanford  Uni¬ 
versity,  September  1991. 

_ ’0]  M.  Genossar  Spectral  Characterizations  of  Nonstationary  Processes,  Ph.D.  Dissertation,  Stan¬ 
ford  University,  April  1992. 


4 


D.  Pal,  Fast  Algorithms  for  Structured  Matrices  with  Arbitrary  Ranh  Profile, 
Ph.D.  Dissertation,  Stanford  University,  May  1Q90. 


Abstract 


Triangular  factorization,  solution  to  linear  equations,  inversion,  computation  of  rank  profile 
and  inertia  (in  the  Hermitian  case)  etc.  of  general  n  x  n  matrices  require  0(n3)  operations. 
For  certain  structured  matrices  including  Toeplitz  and  Hankel  matrices  the  computational 
complexity  is  known  to  be  0(n2)  or  better.  These  structured  matrices  often  arise  in  a 
wide  variety  of  areas  including  Signal  processing,  Systems  theory  and  Communications. 
Fast  (i.e.  0(n2))  algorithms  for  these  structured  matrices  have  been  actively  studied  for 
over  twenty  five  years.  However  almost  all  the  authors  have  assumed  that  the  underlying 
matrices  are  strongly  regular  i.e.  every  principal  submatrix  is  nonsingular.  Although  some 
fast  algorithms  have  recently  been  developed  for  certain  problems  involving  some  of  these 
structured  matrices  which  may  have  one  or  more  zero  minors,  several  other  problems  remain 
unresolved;  also  a  unified  approach  towards  these  problems  is  lacking.  In  this  dissertation, 
we  obtain  several  new  results  through  a  unified  approach  to  the  problems  mentioned  earlier. 

First  we  derive  fast  (i.e.  0(n2))  procedures  for  computing  a  “modified”  triangular  fac¬ 
torization  which  is  a  LDU  factorization  where  L  is  lower  triangular  (resp.  V  is  upper 
triangular)  with  unit  diagonal  entries  and  D  is  a  block  diagonal  matrix  with  possibly  vary¬ 
ing  block  sizes.  Only  strongly  nonsingular  matrices  will  always  have  a  purely  diagonal 
nonsingular  D  matrix.  For  the  matrices  we  study  the  diagonal  blocks  are  also  structured: 
in  particular  they  are  Hankel  (resp.  Quasi- Toeplitz)  for  Hankel  (resp.  Toeplitz)  and  Quasi- 
Hankel  (resp.  Quasi- Toeplitz)  matrices. 


A  particular  application  of  our  result  is  a  fast  method  of  computing  rank  profile  and 
inertia,  leading  to  alternative  proofs  of  certain  results  due  to  lohvidov  (1974)  on  rank  profile 


and  inertia  of  Hankel  and  Toeplitz  matrices. 

Next  using  the  results  on  modified  triangular  factorization  we  extended  the  Schur  com¬ 
plement  based  approach  of  Chun  [Chu89]  for  inversion  of  strongly  regular  Toeplitz  and 
Hankel  matrices  to  Hermitian  Toeplitz,  Quasi-Toeplitz,  Hankel,  and  Quasi-Hankel  matrices 
with  arbitrary  rank  profile. 

We  apply  our  procedures  for  computing  inertia  to  derive  0(n2)  procedures  for  comput¬ 
ing  root  distribution  of  an  n-th  order  polynomial  with  respect  to  the  unit  circle  and  the 
imaginary  axis.  In  particular  it  has  been  possible  to  derive  the  first  general  recursive  pro¬ 
cedure  for  determining  the  root-distribution  of  a  polynomial  with  respect  to  the  unit  circle: 
the  classical  Schur-Cohn  test  for  this  problem  fails  in  the  presence  of  certain  “singularities." 
which  correspond  to  a  Quasi-Toeplitz  Bezoutian  matrix  being  non-strongly-regular. 
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Abstract 


IN  THE  AREA  OF  DETECTION  AND  ESTIMATION,  there  is  a  class  of  so- 
called  signal  subspace  algorithms  that  has  various  applications  in  mobile/cellular 
communications,  target  tracking  and  signal  estimation,  system  identification,  ARMA 
modeling,  adaptive  filtering.  Though  signal  subspace  algorithms  have  been  shown 
to  perform  significantly  better  than  the  traditional  least-squares  methods,  most  of 
them  have  been  used  only  for  off-line  analyses  instead  of  being  realized  on-line.  The 
main  reason  is  that  they  all  incorporate  the  common  key  step  of  estimating  the  signal 
subspace,  called  signal  subspace  decomposition,  which  is  conventionally  achieved  by 
computationally  intensive  eigendecomposition  (ED)  or  singular  value  decomposition 
(SVD).  The  ED  or  SVD  of  an  MxM  matrix  requires  at  least  0(M3)  multiplications 
which  is  quite  significant  for  large  M.  Also  in  many  of  its  more  stable  instantiations, 
the  ED  (or  SVD)  involve  global  communication  and  significant  amounts  of  local  stor¬ 
age,  properties  that  make  VLSI  parallel  implementation  difficult.  Therefore,  the  ED 
(or  SVD)  represents  a  significant  barrier  that  prevents  on-line  realizations.  Orders 
of  magnitude  computational  reduction  of  signal  subspace  decomposition  may  lead  to 
real-time  application  of  various  signal  subspace  algorithms  and  thus  result  in  signifi¬ 
cant  progress  in  the  fields  of  signal  processing  and  communications,  among  others. 

In  this  thesis,  we  present  a  class  of  East  Subspace  Decomposition  (FSD)  algo¬ 
rithms  by  exploiting  the  matrix  structure  associated  with  signal  subspace  algorithms. 
The  new  FSD  techniques,  based  on  the  well-known  Lanczos  algorithm  for  tridiago- 
nalization,  require  only  0(M2d)  multiplications  for  an  MxM  matrix,  where  d  is  the 
dimension  of  the  signal  subspace.  In  the  aforementioned  applications,  M  is  usually 


much  larger  than  d  and  FSD  achieves  an  order  of  magnitude  reduction  in  computa¬ 
tional  complexity.  If  the  matrix  under  consideration  has  additional  structure  common 
in  signal  processing  and  communications,  e.g.,  Toeplitz  and  Hankel,  FSD  can  exploit 
such  structures  and  achieve  another  order  computational  reduction.  New  detection 
schemes  for  estimating  d  are  also  presented  and  can  be  carried  out  in  the  process 
of  signal  subspace  decomposition.  More  importantly,  the  FSD  approach  can  be  eas¬ 
ily  implemented  in  parallel  using  simple  array  processors,  and  the  computation  time 
can  be  reduced  further  to  O(Md)  or  O(logMd)  using  O(M)  or  0(M 2)  multipliers, 
respectively. 

This  thesis  also  presents  some  new  results  in  numerical  linear  algebra,  based  on 
which  the  FSD  is  derived.  In  particular,  a  new  error  estimate  of  the  Rayleigh- Ritz 
approximation,  a  key  step  of  the  Lanczos  algorithm,  is  derived;  this  error  estimate  is 
much  tighter  than  the  well-known  Kaniel  and  Saad  bounds  for  the  matrix  of  inter¬ 
est.  Combining  these  new  results  with  the  theory  of  multivariate  statistical  analysis, 
enables  a  rigorous  performance  analysis  of  the  FSD  approach.  Unlike  many  fast  algo¬ 
rithms  that  trade  performance  for  speed,  the  performance  analysis  shows  that  FSD 
has  the  same  asymptotic  performance  as  its  more  costly  counterparts,  ED  and  SVD. 
and  that  the  new  detection  schemes  are  strongly  consistent. 
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Abstract 


Nonstationary  random  processes  are  encountered  in  many  fields.  Various  approaches 
have  been  suggested  for  defining  spectra  for  nonstationary  processes.  Among  them 
are  the  short  time  Fourier  transform,  the  Wigner-Ville  distribution,  the  asymptotic 
spectrum,  the  cyclic  autocorrelation,  the  cyclic  spectrum  and  Loeve’s  harmonizable 
representation. 

This  dissertation  is  concerned  with  modeling,  characterizing  and  estimating  spec¬ 
tra  for  nonstationary  discrete-time  processes,  in  a  probabilistic  framework.  Our  focus 
is  on  persistent  processes,  and  within  those  we  are  particularly  interested  in  processes 
-hat  contain  some  periodic  structure. 

First  we  develop  the  probabilistic  theory  for  nonstationary  processes  in  the  har¬ 
monizable  framework,  which  is  based  on  a  bivariate  Fourier  transform.  We  discuss 
the  properties  of  various  nonstationary  processes  in  this  framework,  and  show  how 
other  spectral  representations  of  nonstationary  processes  (the  Wigner-Ville  distribu¬ 
tion,  the  cyclic  autocorrelation  and  the  cyclic  spectrum)  fit  into  this  framework.  We 
classify  processes  according  to  their  spectral  support,  and  discuss  decomposition  of 
the  bivariate  spectral  measure. 

Next  we  present  a  statistical  analysis  of  an  estimator  of  the  cyclic  autocorrelation 
for  nonstationary  Gaussian  processes.  We  derive  new  necessary  and  sufficient  condi¬ 
tions  for  consistency  in  mean  square  of  the  estimator,  and  discuss  rate  of  convergence 
of  the  estimator. 

Finally  we  consider  models  and  spectral  relations  for  cyclostationary  processes 
Gladyshev  has  suggested  a  correspondence  between  cyclostationary  processes  and 


multichannel  stationary  processes.  For  this  correspondence  we  derive  relations  be¬ 
tween  the  spectral  representations  of  the  cyclostationary  process  and  of  the  multi¬ 
channel  stationary  process.  This  provides  ties  between  the  theory  of  multichannel 
stationary  processes  and  the  theory  of  cyclostationary  processes.  Periodic  autore¬ 
gressive  and  periodic  autoregressive  moving-average  models  are  sometimes  used  to 
model  cyclostationary  processes.  We  derive  relations  between  the  parameters  of  such 
models  and  the  cyclic  spectrim  of  the  cyclostationary  process.  These  relations  can 
be  used  for  parametric  estimation  of  the  spectrum  of  cyclostationary  processes. 


